Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Optimise the CI pipeline for faster runs and earlier failures #1869

Merged
merged 8 commits into from
Mar 12, 2025

Conversation

jacksonj04
Copy link
Collaborator

@jacksonj04 jacksonj04 commented Mar 6, 2025

Improve our build process with two goals:

  • Don't perform unnecessary work if we can fail earlier; add prerequisite steps so that CI follows a logical pipeline
  • Leverage GitHub Actions cache so that Docker can do less work rebuilding images from scratch each time

by:

  • Making linter have to run before anything else
  • Run e2e_tests only after both JS and SCSS have completed
  • Using the GitHub Actions cache for Docker layer caching
  • Making e2e_tests run after pytest
  • Only build our Django image once and reuse it between test steps

@jacksonj04 jacksonj04 force-pushed the speedy-tests branch 20 times, most recently from a784177 to 1d66a8e Compare March 12, 2025 12:42
@jacksonj04 jacksonj04 changed the title Improve the speed of tests by reusing Docker build outputs Optimise the CI pipeline for faster runs and earlier failures Mar 12, 2025
@jacksonj04 jacksonj04 force-pushed the speedy-tests branch 9 times, most recently from 6c081b0 to 304c916 Compare March 12, 2025 15:00
@jacksonj04 jacksonj04 force-pushed the speedy-tests branch 5 times, most recently from ff97768 to a437ca8 Compare March 12, 2025 15:36
@jacksonj04 jacksonj04 requested a review from dragon-dxw March 12, 2025 15:36
@jacksonj04 jacksonj04 marked this pull request as ready for review March 12, 2025 15:36
This will help us fail fast and prevent unnecessary builds for code which would fail basic linting.
There's no point running e2e tests if we're not confident other parts of the application are working.

When we get proper caching in place this will run in series after pytest as well, but for the moment we keep it in parallel to keep CI runtimes down.
This is now the default, it's just adding noise to the build process.
This should dramatically speed up the process of building the stack since unchanged layers can be reused from previous build operations.
We build the exact same image for both pytest and e2e, so use it in both.
@jacksonj04 jacksonj04 force-pushed the speedy-tests branch 2 times, most recently from e1d5439 to bb1523c Compare March 12, 2025 15:52
This was trying to build all services, but we don't actually need things like the local docs for this. Only launch `django` and its dependencies.
@jacksonj04 jacksonj04 added this pull request to the merge queue Mar 12, 2025
Merged via the queue into main with commit c39eb76 Mar 12, 2025
18 checks passed
@jacksonj04 jacksonj04 deleted the speedy-tests branch March 12, 2025 16:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants